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Summary 

This report documents the state of the art of ocu- 
lometric data analysis techniques and their applica- 
tions in certain research areas such as pilot work- 
load, information transfer from displays to pilots, 
strategy and role of pilots, and pilot training. The 
analysis techniques result in the following data: real- 
time viewing of the pilot’s scanning behavior, aver- 
age dwell times, dwell percentages, instrument tran- 
sition paths, dwell histograms, and entropy rate mea- 
sures. Visual workload estimates are obtained from 
dwell percentages and entropy rate measures. Infor- 
mation transfer evaluations are performed primarily 
with average dwell times, dwell histograms, and en- 
tropy rates. Pilot strategy and role are determined 
by using scan time histories, time-locked time histo- 
ries, and transition matrix data. Pilot training eval- 
uations use real-time scanning data and entropy rate 
measures. Overviews of the experimental setup, data 
analysis techniques, and software are presented. Sev- 
eral results from these techniques are discussed. A 
glossary of terms frequently used in pilot scanning 
behavior and a bibliography of reports on related re- 
search sponsored by NASA Langley Research Center 
are presented. 

Introduction 

Man-machine flight research at Langley Research 
Center (LaRC) addresses human factors problems 
such as the effects of a particular type of instrument 
on pilot workload, the effects of a new instrument 
on pilot strategy, the effects of a new procedure on 
pilot role, and the effects of data format on infor- 
mation transfer. To investigate these problems, a re- 
search program was initiated to develop experimental 
protocols and techniques for analyzing data on pilot 
scanning behavior. 

The measurement of pilot scanning behavior be- 
gan in 1946 when Jones et al. (ref. 1) determined a 
pilot’s lookpoint by subjectively judging motion pic- 
ture frames of a pilot’s face. The outcome of this 
early research was the development of an instrument 
placement standard (namely, “the standard six” ) and 
shape-coding for control knobs. The results of that 
work are still being used as design standards and can 
be found in a number of handbooks (e.g., ref. 2). The 
results were also responsible for the FAA’s regula- 
tions concerning the basic “T” arrangement of elec- 
tromechanical instruments (ref. 3). These design 
standards, which are still being applied today, are 
helpful, but they may not be sufficient for designing 
current and future aircraft cockpit displays. 

With the advent of the oculometer, progress has 
been made in the past decade in the development of 


the necessary scanning-behavior analysis techniques, 
coupled with their correlation to control inputs and 
aircraft parameters, to evaluate display design. In 
the past it had been erroneously assumed that if 
dwell times and scanning sequences could be quanti- 
fled, then a better display could be designed. These 
scanning-behavior measures by themselves do not 
provide sufficient information for the optimal design 
of a display. Today, with the aid of oculometer hard- 
ware and software, determining in real time where a 
pilot is looking can be done to an accuracy of 1 visual 
degree (a circle 0.5 in. in diameter). These data are 
recorded in a computer-compatible format 30 times 
per second along with other pertinent aircraft and 
performance measurements. The oculometer hard- 
ware has been developed and miniaturized to the 
point that it can be placed in most aircraft simulator 
cockpits. For the past 10 years, NASA has been de- 
veloping analytical tools which when applied to pilot 
scanning-behavior data will give measurements that 
can be used to interpret pilot workload, to determine 
relative rates of information transfer from displays to 
pilots, to determine pilot strategy and/or role, and 
to assist in pilot training. 

The purpose of this report is to review the tech- 
niques developed at LaRC for analyzing data on pi- 
lot scanning behavior and to show how these tech- 
niques can be applied to the human factors research 
areas mentioned above. Four appendices are included 
in the report. Appendix A, written by Daniel W. 
Burdette of PRC Kentron, Inc., describes the evolu- 
tion of LaRC’s oculometer system over the past sev- 
eral years as well as procedures for its setup and cal- 
ibration. Appendix B provides a functional descrip- 
tion of three computer programs which have been de- 
veloped to analyze data on pilot scanning behavior. 
Appendix C is a glossary of frequently used terms in 
scanning behavior. Appendix D is a bibliography on 
scanning-behavior research sponsored by LaRC. 

Symbols and Abbreviations 


AC 

alternating current 

ADF 

automatic direction finder 

AGL 

above ground level 

AS 

airspeed indicator 

BA 

barometric altimeter 

CMD 

command 

D 

number of off-diagonal terms in 
transition matrix 

DT 

average dwell time 

EADI 

electronic attitude direction indicator 



EHSI electronic horizontal situation 
indicator 

FD flight director 

GSI glide slope indicator 

H average entropy 

H entropy rate 

HSI horizontal situation indicator 

ILS instrument landing system 

N number of instruments 

P dwell percentage 

Pij two-way transition percentage between 
instruments i and j 

p probability 

RA radar altimeter 

TD task demand 

VSI vertical speed indicator 

X element in a matrix 

Subscripts: 

i “from” instrument 

3 “to” instrument 

Anaytical Procedures 

The analysis techniques which have been devel- 
oped to analyze pilot scanning behavior are presented 
in four sections. The first section describes qualita- 
tive evaluations of the pilot’s lookpoint on the in- 
strument panel. The second section describes the 
somewhat traditional quantitative analysis of time 
histories, time-locked time histories, transition ma- 
trices, and statistical tests of scanning-behavior mea- 
sures. The third and fourth sections describe more 
advanced quantitative analysis techniques of dwell 
histograms and measurement of scanning random- 
ness. Other analysis tools, based upon the authors’ 
experience, that could be developed are proposed in 
a fifth section. 

Real-Time Viewing of Lookpoint 

A very simple method of analyzing scanning be- 
havior is observing the video of the instrument panel 
scene with the pilot’s instantaneous lookpoint super- 
imposed upon it. Figure 1 is a composite photograph 
showing the measurement equipment, the pilot flying 
a simulator, and the type of video presentation avail- 
able to the researcher conducting the tests. Viewing 
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Figure 1. Oculometer setup with real-time view of subject’s 
lookpoint. 


the instantaneous lookpoint can be done in real time 
during simulated or actual flight or from a video tape 
after the testing session is over. Observing the pilot’s 
lookpoint as it moves from instrument to instrument 
enables the researcher to quickly develop an impres- 
sion of the pilot’s scanning behavior. In addition, 
viewing the lookpoint provides some appreciation of 
the quality of the data being collected during the 
experimental sessions. For example, the researcher 
can detect consistent dwells skewed off the instru- 
ment faces indicating poor calibration, or he may 
observe extended “out-of-track” periods indicating 
that certain parameters should be checked and/or 
readjusted. (See appendix A.) The real-time video 
also provides the operator with information needed 
to assure that the oculometer system is operating 
correctly. For example, if the system is out of track 
for extended periods of time, then certain oculometer 
system parameters should be checked. 

Traditional Scanning-Behavior Analysis 

One of the first questions asked about a pilot’s 
scanning behavior in many circumstances is, how 
much time does a pilot spend on the various in- 
struments? The software required for providing the 
answer would seem to be quite straightforward and 
simple. However, it can be quite complex, depend- 
ing upon the type of scanning-behavior statistics that 
are calculated and the sophistication of the algorithm 
used to determine fixations. The data analyses pre- 
sented here include time histories, time-locked time 
histories, transition matrices, and statistical tests 
such as the t test and the F ratio. 

Time histories. A simple method of exploring the 
question of where a pilot looks can be answered by a 
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Figure 2. 

approach (altitude of 700 to 100 ft above ground level). 

time history plot of the lookpoint. A typical time his- 
tory is shown in figure 2. The ordinate is scaled in di- 
visions representing the different instruments. Time 
is plotted along the abscissa. This technique, like 
real-time viewing, permits a subjective evaluation of 
the scanning behavior by allowing the researcher to 
see the entire run in one view and thereby identify 
some candidate measures for scanning behavior, such 
as (1) the number of times a pilot looks out of the 
window on a landing approach, (2) the number of 
times a pilot looks at the engine instruments on take- 
off, (3) the number of blinks per minute, or (4) the 
total number of transitions between any particular 
pair of instruments. Particular analytical evaluation 
procedures can then be incorporated into computer 
programs, and in turn, the reduced data can be used 
for more detailed evaluation by the experimenter. 

Time-locked time histories. It is useful to know 
how a pilot responds to various tasks which he must 
perform. One recently developed analysis technique 
involves time locking the scanning analysis to some 
event, such as performing a secondary task. This 
technique is similar to that used in the analysis of 
evoked brain potentials in reference 4, where the am- 
plitudes of the brain waves at specified times from the 
beginning of triggering events are averaged. A time 
history is produced in which the ordinate values rep- 
resent the average amplitude and the abscissa is the 
time with respect to the triggering event. In this 
scanning-behavior analysis procedure, however, the 
percent of time looking at each instrument at speci- 
fied times relative to triggering events (for example, 
a beep) is calculated. A time-locked time history of 
these dwell percentages is produced in which the or- 
dinate values represent the percent of time spent on 
each instrument and the abscissa is the time with 
respect to the triggering event. Figure 3 illustrates 
in bar graph form what the plot would look like at 
a given sample time relative to the triggering event. 
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Figure 3. Time-locked time history at a given sample time 
relative to a triggering event. 


These percentage values are obtained by adding look- 
points which occurred at the same time relative to 
many occurrences of the same triggering event. The 
height of each bar is proportional to the dwell per- 
centage of the appropriate instrument at a time rel- 
ative to the triggering event. For example, the dwell 
percentage on the attitude indicator in figure 3 would 
be approximately 55 percent. By plotting all the in- 
struments in the same relative vertical positions and 
placing all the oculometer samples side by side, a 
time history is formed. (See fig. 4.) 



Figure 4. Time-locked time history. 


Figure 4 shows data collected during tests to 
determine the dwell times necessary to accomplish 
specific information-gathering tasks. The dwell time 
taken by a pilot to classify the up or down position 
of a display needle whenever a beep sounded was 
determined. The time history was time locked to 
a beep (time zero) that signaled the start of the 
secondary task. In this figure each instrument is 
represented by a different shading. The time interval 
corresponds to the sampling rate of the oculometer 
(30 samples per second). As seen in the figure, about 
one third of a second after the beep, the beginning of 
the transition to the side task is initiated (increase in 
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percentage of time on the side task). The first event 
after the look at and response to the side task is a 
blink (as indicated by the momentary loss of track by 
the oculometer), followed by a look at the attitude 
indicator, then the directional gyro (both indicated 
by increases in the percentage of time looking at these 
instruments). Events that are naturally occurring in 
a simulation (e.g., pressing of a particular button 
by a pilot, a sound from the master caution and 
warning system, or receipt of a certain instruction 
from air traffic control) could be used to trigger the 
data instead of a contrived beep. 

Transition matrices. The transition matrix de- 
scribes the probability of transitioning from one in- 
strument to another and can be thought of as a 
second-order Markov transition matrix. Each ele- 
ment of the transition matrix is the total number of 
one thirtieths of a second that the pilot looked at 
an instrument during an oculometer sample period, 
given that he was looking at some other or the same 
instrument during the previous oculometer sample. 
These matrices are used to quantify the time a pilot 
spends on the various instruments and the patterns 
of his eye movements. These measures include av- 
erage dwell times, instrument dwell percentages, and 
one-way or two-way transitions (see appendix C) and 
can be quite useful when performing exploratory or 
descriptive research tests. 

For the situation of three instruments, the tran- 
sition matrix would be as follows: 

“To” instrument number, j 


“From” instrument number, * 


1 

2 

3 

Xll 

^12 

X13 

X21 

X22 

X2S 

^31 

XS2 

^33 


The first subscript to the matrix element represents 
the “from” instrument, (f), and the second subscript 
represents the “to” instrument, (j). The result after 
analyzing a set of data is that each X{j has the value 
of the number of times the transition from instrument 
i to instrument j occurred. The magnitudes of the 
diagonal elements are the largest, since more time 
is spent looking at an instrument (transitions to the 
same instrument) than is spent transitioning to new 
instruments. From the data in this matrix, several 
measures can be calculated. 

Average dwell time of each instrument can be 
obtained by summing the elements of each column 
and dividing by the total number of transitions to 
that instrument (sum of the off-diagonal terms in 
that column) and converting the quotient to seconds 


by dividing by the sampling rate of the oculometer 
(30 samples per second). The average dwell time for 
instrument j, (DT)y, would be 


(DT), = 

^ 30 


where N is the number of instruments. 

The dwell percentage on each instrument can be 
calculated by dividing the sum of each column by the 
total sum of all the elements. For instrument j in the 
above example, the dwell percentage, Pj, would be 


^ r • • 

^ 3—1 •‘'13 

Finally, the occurrence of a specific one-way transi- 
tion as a percentage of all one-way transitions can be 
calculated by dividing the appropriate off-diagonal 
element by the sum of all the off-diagonal elements. 
Two-way transition percentages, that is transition 
percentage from instrument 1 to 2 (^ 12 ) and from 
instrument 2 to 1 ( 3 : 21 ), can be obtained by sum- 
ming the appropriate off-diagonal elements (i.e., X 12 
and 3 : 21 ) and dividing by the sum of the off-diagonal 
terms. Thus, the two-way transition percentage, P 12 , 
between instruments 1 and 2 would be 


P12 


^12 + X21 


2^i=l 2 ^ 7=1 
j^i 


Xij 


Graphs showing these transition matrix measures 
are shown in figures 5 and 6. These figures are 
airline pilots’ scans during an instrument landing 
approach (ref. 5). To assist in the interpretation 
of the data, these graphs approximate the placement 
and shape of instruments on the instrument panel. 
Generally, the instruments are thought of as round, 
since their faces are circular, even though the outside 
bezel is almost square. Inside the outline of each 
instrument are the dwell percentage, rounded off 
to the nearest percent or tenth of a percent, and 
the average dwell time, rounded off to the nearest 
tenth or hundredth of a second. Lines between 
the instruments show the transition paths that were 
found in the data, with the two-way transition values 
indicated. The thickness of the line connecting two 
instruments is drawn proportional to the value of the 
two-way transition percentage to facilitate a more 
rapid visual interpretation of these transition paths. 
Gompare the dwell percentages on the flight director 
during ILS approaches (figs. 5 and 6) for manual 
and automatic (coupled) flight modes. Figure 5 
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(b) Coupled ILS. 

Figure 5. Manual and coupled ILS approaches with no 
atmospheric turbulence. Transition values less than 
2 percent were omitted. 




(b) Coupled ILS. 

Figure 6. Manual and coupled ILS approaches with atmo- 
spheric turbulence. Transition values less than 2 percent 
were omitted. 

shows data with no turbulence, and figure 6 shows 
the same data with turbulence. Slightly more of 
the pilot’s attention (dwell percentage on the flight 


director) was required with turbulence than with no 
turbulence. 

Appendix B gives a functional description of three 
computer programs which were developed to analyze 
pilot scanning data by calculating the transition ma- 
trices. These programs are SCAN, SUMMARY, and 
HISTO. 

Statistical tests of scanning-behavior measures. 

Differences in independent factors (testing condi- 
tions) can be evaluated by using scanning-behavior 
measures (average dwell time, dwell percentage, and 
two-way transition probability) as dependent vari- 
ables. One problem with these statistical tests in 
the past has been a general lack of consistency in 
the evaluations from one condition to another. This 
problem has generally been overcome by the collec- 
tion of enough data to produce consistent trends. A 
second problem, the interpretation of changes in the 
scan measures, has been alleviated by more basic re- 
search into scanning behavior and the development 
of more advanced analysis techniques. A third prob- 
lem with the average dwell time measure is a lack 
of sensitivity. For instance, the average dwell time 
has a large standard deviation, almost equal in mag- 
nitude to the average, whether it is measured from 
one pilot or several. This lack of sensitivity has been 
overcome by the use of the dwell histogram and the 
correlation of dwells with pilot activity. (See the sec- 
tion on “Dwell Histograms”.) 

The computer program summary, described in 
appendix B, is used to perform standard statisti- 
cal tests. The output of the summary program 
consists of four tables, including tables of sums of 
squares, averages, F ratios of variances, and two- 
sample t-test values and the associated degrees of 
freedom. The values in these tables are used to deter- 
mine whether statistically significant differences exist 
between conditions. 

Dwell Histograms 

The answer to the question of why a pilot looks 
at a display the way he does can be inferred from a 
histogram of the dwell times on that instrument. A 
dwell histogram is a graph of either the total num- 
ber of dwells or the percent of total dwells on an 
instrument which lasted for the time indicated by 
the abscissa. Intuitively, the length of the dwell time 
should be inversely related to the rate of information 
transfer from the display to the pilot. Longer dwells 
indicate that more information is being taken in (see 
the discussion on differences between monitoring and 
controlling dwells later in this section), that the dis- 
play is slower in transferring the information (refs. 6 
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and 7), or that the pilot is staring at the display. 
At the present time it is left to the judgment and 
expertise of the experimenter to determine whether 
the longer dwell time is reflecting slower information 
transfer, an increased amount of information trans- 
fer, or staring. 

To demonstrate that the dwell histogram is sen- 
sitive to the rate of information transfer in a display, 
tests in which the format of the directional gyro was 
altered were conducted in the Langley General Avi- 
ation Simulator (ref. 7). One format used a con- 
ventional “compass rose”; the other format used a 
compass rose with a movable “bug” which could be 
positioned at the desired aircraft heading. It is obvi- 
ous that the bug is an enhancement to the heading 
display because with the bug the pilots are able to 
see the heading error, as indicated by the position of 
the bug relative to the flxed pointer, instead of hav- 
ing to estimate their heading and mentally compare 
it with the desired heading of the aircraft. The dwell 
histograms of the two types of display are presented 
in flgure 7. As expected, the dwell histogram with 
the bug is further to the left (shorter dwell times) 
than the dwell histogram without the bug. These 
results show that the dwell histograms can be use- 
ful in evaluating information transfer rate of display 
formats. 



Figure 7. Directional gyro-monitoring dwell histogram. 



Figure 8 . Dwell distributions. 


Dwell time has a large standard deviation even if 
it is measured from just one test subject. One of the 
first attempts to evaluate the cause of the large vari- 
ations in dwell times was a correlation of the dwells 
with certain pilot activities, such as making control 
inputs, and the examination of the dwell histogram 
associated with each activity (ref. 8). The result- 
ing dwell histograms show several different distribu- 
tions. Figure 8 shows four dwell distributions with 
peaks at 0.1, 0.4, 0.6, and 1.2 sec. When plotted 
on a logarithmic scale as in this figure, the curves 
appear symmetrically distributed about their peaks. 
These curves are all significantly different (p < 0.01) 
when evaluated with the Kolmogorov-Smirnov test 
(ref. 9). The two distributions with the shortest du- 
ration are for monitoring dwells (dwells during which 
no control input was made), and the two distribu- 
tions with the longest durations are for control in- 
put dwells (dwells during which control inputs were 
made). It is hypothesized that the monitoring dwell 
histogram with a peak at 0.1 sec is associated with 
“subconscious” (nonrecallable) verification of infor- 
mation (i.e., a glance), while the monitoring dwell 
histogram with a peak at 0.4 sec is associated with 
“conscious” (recallable) reading of information. The 
control input dwell histograms are classified accord- 
ing to the number of control inputs made during that 
dwell. As seen in the flgure, dwells longer than 1.0 sec 
are almost eliminated from the monitoring dwell 
classification. 

Closely associated with the dwell histogram is the 
fixation histogram. A dwell on an instrument is made 
up of the series of continuous fixations which re- 
main within the boundary of that instrument. Even 
though the average dwell time parameter is not a 
sensitive statistical measure, the fixation time his- 
tograms are very consistent from pilot to pilot with 
similar skill levels and, when separated according to 
pilot activity such as monitoring or controlling, seem 
to be insensitive to environmental factors, such as 
turbulence, or situational factors, such as the maneu- 
ver being performed. Turbulence is associated with 
a reduction in the number of fixations that are linked 
together to make a dwell rather than with a change 
in the length of the individual fixations. In one series 
of tests in a fixed-base simulator in which the effect 
of turbulence was evaluated, the number of fixations 
per dwell was reduced, as seen in table I. It appears 
that the effect of turbulence (or an increase in the 
frequency content of displayed information) is a re- 
duction in the amount of information gathered only 
while making control inputs. It could be that the pi- 
lot abandons the verification that his input has had 
the intended effect and instead transitions to another 
instrument looking for additional information. 
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TABLE L FLIGHT DIRECTOR FIXATIONS PER 
DWELL WITH AND WITHOUT TURBULENCE 



Fixations per dwell 

Type of dwell 

Turbulence 

No turbulence 

Monitoring 

2.0 

2.0 

Controlling 

4.0 

7.0 


Timing of control inputs. Display effects on scan- 
ning behavior involve more than just how long a pilot 
looks at a given display. The format and ease of in- 
terpreting a display may have a noticeable effect on 
the dwell time on the display scanned next, partic- 
ularly when the pilot is required to mentally assimi- 
late the information gained from the first display. In 
some cases, dwell time effects may be so subtle that 
they are not evident in the average dwell time on 
that instrument. For example, the dwell time on the 
flight director is already influenced by a variety of 
factors, as evidenced by its large average and stan- 
dard deviation. However, effects on the dwell time 
may be evident in the average duration of the atti- 
tude dwell following a look at the display of interest. 
These carry-over dwell time effects were present in 
tests reported in reference 10, in which an all-digital 
altimeter was compared with a conventional counter- 
drum-pointer altimeter. The average dwell time on 
the attitude indicator following a dwell on the digi- 
tal altimeter was increased over the average attitude 
dwell time after looking at the counter-drum-pointer 
altimeter. Moreover, when control inputs were made 
during the attitude dwell after the sequence altitude 
dwell-attitude dwell, it took longer to make a control 
input when the altimeter was digital. The two his- 
tograms of the timing of the control input are shown 
in figure 9. Pilots’ comments support these oculo- 
metric data by revealing that they had to think about 
altitude more with the digital altimeter than with the 
counter-drum-pointer altimeter. Consequently, the 
pilots took longer to assimilate and integrate altitude 
information with the digital altimeter than with the 
counter-drum-pointer altimeter. 

Statistical tests of histograms. It is quite de- 
sirable to determine if two histograms are statisti- 
cally different from each other because of differences 
in test conditions. A nonparametric test which can 
be used for histograms is the Kolmogorov-Smirnov 
test (ref. 9). This test is used to find the maximum 
difference between two cumulative distributions (the 
sum of all histogram levels from the lowest to the 
current point of interest) and based upon the num- 
ber of data points in both curves, to determine at 


O Counter-drum-pointer dwell 

(Average attitude dwell = 0. 60 sec) 



Figure 9. Pitch control input timing during attitude dwell 
following an altitude dwell. 


what significance level the two are different. Thus 
far, this testing procedure has produced results with 
“face validity,” that is, the results of the test are 
what one would have predicted by looking at the raw 
histograms. 

Measurement of Scanning Randomness 

Markov probability transition matrices (refs. 11 
and 12) describe the predominant patterns in the 
scan by means of the relative sizes of transition prob- 
abilities, but comparison of two of these matrices for 
different experimental conditions has proved to be 
either extremely difficult or impossible. Some re- 
searchers (refs. 11 and 12) have used a chi-square test 
of the transition matrices; however, equivocal results 
were obtained because some simplifications and ma- 
nipulations of the data were made which are not sup- 
ported by any theoretical basis. A goal of scanning- 
behavior research is to identify a general method for 
the study of scanning behavior. This method should 
be independent of the number and arrangement of 
instruments. The nature of oculometric lookpoint 
data suggests some methods from information the- 
ory (ref. 13) which may have this generality (ref. 14). 

The term entropy has been associated with infor- 
mation theory for so long that its usage tends to sug- 
gest an attempt to quantify the information content 
of some system. However, in an even older usage of 
the term in thermodynamics, entropy is used to de- 
scribe the amount of disorder (randomness) present 
in a system. In the present discussion it must be 
emphasized that the attempt is not to quantify the 
amount of information which the pilot is acquiring 
from the displays, but rather to quantify the amount 
of spatial and/or temporal randomness present in the 
pilot’s scan behavior. This concept coincides with the 
meaning of entropy in thermodynamics. 

The piloting task is such that the pilots’ scan is 
limited to a specified number of instruments {N) at 
a time. The time history of dwells has a form similar 
to that of a communication system which can assume 
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one of N discrete states with a varying duration in 
each state. The orderliness of such a system is related 
to the probabilities with which the system occupies 
its different states. A system that always occupies 
the same state or always makes the same transitions 
between states is not random. In the case of instru- 
ment scan, these situations would be paralleled by 
staring or by stereotyped scan patterns. This con- 
cept of system order may be stated compactly by us- 
ing information theory (ref. 13) to generate a mathe- 
matical equation for entropy of letter sequences and 
applying it to instrument sequences scanned. The 
entropy of the scan becomes 

D D 

^ = Pi 

i=l i=l 

where 

H observed average entropy 

Pi probability of transition between two 

instruments X{j divided by the sum of the 
off-diagonal terms in the transition matrix 

D number of off-diagonal terms in transition 
matrix N{N — 1) 

N number of instruments 

To calculate the entropy, first convert the off- 
diagonal terms of the transition matrix to percent- 
ages by dividing by the sum of the off-diagonal terms. 
Entropy is the overall sum of the products of each 
nonzero off-diagonal terms and the base 2 log of the 
same off-diagonal term of the transition matrix as 
follows: 

H = - (xi2 log2 Xi2 -I- a:i3 log2 2:13 
+ 2^21 log2 2:21 + X23 log2 2:23 
+ 2:31 log2 2:31 -I- X32 log2 2:32) 

In the case of scanning behavior, entropy has the 
units of bits per transition and provides a measure 
of the randomness of the scan behavior. The lower 
the numerical value of the entropy, the less random 
(more ordered) is the scan pattern. The maximum 
possible entropy is constrained by the number of 
instruments and the number of nonzero off-diagonal 
terms. The entropy measure uses the probabilities 
which are present in the Markov transition matrices, 
but it yields a single, more compact expression of 
the overall behavior of the probabilities rather than 
presenting them individually. 

While entropy is a measure of the randomness of 
the scanning behavior, it does not take into account 


the differences in dwell time for each dwell. Dwell 
time can be markedly affected during high mental 
workload situations. To include the effect of time, 
entropy rate H is defined as 



D 

H ~ 

h 

where 



entropy for transition i as given above 

(DT),- 

average combined dwell time for instru- 
ments when transition i occurs 

D 

number of off-diagonal terms in transition 
matrix, N[N ~ 1) 

N 

number of instruments 


Entropy rate, like entropy, indicates randomness 
(in the sense of rate of change in randomness) in the 
scan pattern. Entropy rate analysis was performed 
on scanning-behavior data of tests conducted to eval- 
uate two types of vertical speed indicators, VSI’s 
(ref. 6). These tests and others (ref. 14 ) demon- 
strated that as the level of mental loading required 
for a secondary task increases, the entropy rate de- 
creases exponentially (fig. 10, taken from ref. 6). In 
other words, the scan was more ordered (i.e., less ran- 
dom) with higher mental loading and resulted in a 
lower value of entropy rate. Therefore, in comparing 
two systems or displays, the system or display with 
the lower entropy rate (more ordered scan pattern) 
would indicate a higher mental workload. The expla- 
nation for this effect on entropy rate is that the higher 
mental loading deprives the pilot of the time needed 
to make the normal cross checks (i.e., random looks) 
at other instruments; therefore, he concentrates his 
scanning on the essential instruments. 

The entropy rate for a VSI with a vertical bar 
graph display format (vertical VSI) and a conven- 
tional VSI at different imposed mental workload con- 
ditions is plotted in figure 10. The entropy rate, 
measured in bits per second, of the vertical VSI was 
greater than that of the conventional VSI. Since in- 
creasing the imposed mental workload (task demand) 
decreases the entropy rate, the curve with the lower 
entropy rate would indicate the higher workload con- 
dition. Therefore, using the conventional VSI im- 
poses the higher workload. An estimate of the mag- 
nitude of this difference in workload can be obtained 
by finding the value of task demand (TD) in the equa- 
tion for the vertical VSI curve which gives the same 
entropy rate as the conventional VSI at the zero TD 
level. This TD value corresponds to flying with the 
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RATE OF PRESENTATION, triplets per sec 
(TASK DEMAND) 

Figure 10. Entropy rate as a function of mental loading. 

vertical VSI and performing the mental side task at a 
level of 0.045 secondary tasks per second for the ILS 
landing task. Thus, the difference in visual work- 
load is equivalent to performing the mental loading 
secondary ta.sk at a rate of slightly less than 3 per 
minute. 

Other Possible Analysis Methods 

The analysis of scanning-behavior data is just in 
its infancy. Consequently, there are several phenom- 
ena which have not yet been investigated in the con- 
text of flight simulation. They are discussed in the 
following sections. 

Blinking rate. What affects the timing or rate at 
which a pilot blinks? Observation of scanning data 
would seem to indicate that in some circumstances 
blinking is slowed down when the task is getting more 
critical (ref. 15). It is not known if this phenomenon 
is peculiar to a few pilots or if it is applicable to all 
pilots. It is also not known if there are situations in 
which the reverse is true, that is, increased blink rate 
with increased stress. 

Pupil diameter. Pupil diameter has been observed 
to be a function of mental processing of information. 
However, in scanning-behavior studies thus far, only 
one study (ref. 16) has noted a consistent trend in 
pupil diameter (i.e., increase in pupil diameter as the 
landing approach proceeded). Many factors other 
than mental stress affect pupil diameter. Examples 
are ambient light levels and differences in reflectivity 


of instruments. (Some are highly absorbent, others 
are highly reflective.) 

Transition path optimization. One method of 
data analysis which should be pursued is transition 
path optimization. This would be useful in evalu- 
ating graphic displays to determine which symbols 
to locate adjacent to each other and which ones to 
place in the periphery (because they are used less 
frequently). 

Application of Scanning-Behavior Analysis 
Techniques 

Scanning behavior and the previously mentioned 
analysis techniques can be applied to a variety of 
research issues. The following research issues will be 
discussed individually: visual workload, information 
transfer of displays, pilot strategy and role, and pilot 
training. 

Visual Workload 

The term workload incorporates all factors which 
contribute to the level of effort expended by a per- 
son to accomplish a task. There is no single mea- 
sure of total workload, but generally, it is thought 
that workload can be broken down into at least three 
categories: physical workload, mental workload, and 
psychological stress. These categories can be fur- 
ther subdivided into many other factors which affect 
workload. For example, scanning behavior is primar- 
ily related to the mental aspects of workload, that is, 
the time it takes a pilot to acquire information from 
displays. For the sake of clarity in this report, this as- 
pect of workload will be called visual workload. One 
approach is to measure relative amounts of visual 
workload. Scanning-behavior measures which have 
been shown to be useful in evaluating visual workload 
are transition matrix measures (dwell percentage and 
average dwell time) and entropy rate. 

Transition matrix measures. Visual workload 
measures derived from the transition matrix are 
dwell percentage and average dwell time. The longer 
the time spent on an instrument (as measured by 
either dwell percentage or average dwell time), the 
greater the visual effort that is expended to obtain 
information from that particular instrument. This 
assumes, however, that the same types or amounts of 
information are being obtained from the instruments. 
Therefore, it would be inappropriate to compare the 
dwell percentages and/or average dwell times of an 
attitude indicator with those of a flight director be- 
cause more information is displayed in the flight di- 
rector. However, it would be proper to evaluate 
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the differences in average dwell times or dwell per- 
centages of circular meter movements, vertical meter 
movements, and digital meter formats for a particu- 
lar display such as airspeed, directional gyro, or rate 
of climb indicator. 

Entropy rate measures. Another measure which 
has been shown to be useful in estimating visual 
workload is entropy rate (a measure of randomness 
in the total scan pattern). This measure has been 
used with desktop as well as the more sophisticated 
simulators and seems to be generally applicable for 
comparing two different instruments in the instru- 
ment cluster while a single task is being performed. 
It has not worked in simulations of complex tasks 
in which the instrument evaluated was used primar- 
ily during only a small fraction of the entire flight. 
Properly designed flight tasks are necessary for the 
use of entropy rate analysis when comparing two 
instruments. 

Entropy rate measures of novice pilots have been 
shown to be lower than those of fully trained pilots 
(ref. 17). (Presumably the novice pilots had to work 
harder than the fully trained pilots.) After tr ainin g, 
their entropy rates were the same as those of the fully 
trained pilots. (See the discussion on training.) 

Information Transfer 

Histograms can be used to evaluate the rate of 
information transfer from the display to the pilot. If 
a pilot accomplishes a task with the same level of 
performance with a display having a shorter average 
dwell time, then either the information is transferred 
from the display more quickly or less information is 
transferred from the display. The researcher should 
be guided by experience and pilot comments in de- 
ciding which situation is occurring. An obvious ex- 
ample is the difference in dwell percentages with an 
attitude indicator (50 percent) and a flight director 
(75 percent) during ILS approaches. Dwell percent- 
ages on the flight director are much greater because 
of the extra displayed information (command bars 
and raw glide slope and localizer). However, in the 
case of a digital light-emitting diode (LED) altimeter 
and a counter-drum-pointer altimeter, the reason for 
the lower dwell time on the digital altimeter is not 
quite as apparent. Pilot comments indicate that less 
information can be obtained from the digital display 
because of difficulty in perceiving a relative needle 
position or in obtaining rate clues. 

Dwell histograms. The dwell histogram analysis 
procedure was used to analyze data collected in a six- 
degree-of- freedom motion-base simulator (ref. 18). 


The tests were designed to determine the effects of 
correct pitch motion, reversed pitch motion, or no 
motion on pilot scanning behavior in a single-axis 
tracking task. The results are shown in figure 11 
as cumulative fixation histograms. Kolmogorov- 
Smirnov tests of the three conditions indicated that 
the two motion conditions were not statistically dif- 
ferent from each other (p < 0.10) and that both 
motion conditions (correct and reversed) were statis- 
tically different from the no-motion condition (p < 
0.01). These data, while very interesting, do not by 
themselves mean that high-fidelity motion is or is 
not essential. What it does suggest is that there is 
a cue provided by motion (most likely motion onset 
cue) that allows the pilot to shorten his dwells over 
what would be required in a nonmotion simulator 
to detect motion purely by instrument readouts. It 
also suggests that in terms of visual workload, any 
study performed in a fixed-base simulator will pre- 
dict longer dwell times and slower transition rates 
than one performed in a motion-based simulator. 

Entropy rate. Entropy rate can also be used to 
evaluate the information transfer rate of displays. 
(See the discussion in the “Visual Workload” sec- 
tion.) Lower entropy rate indicates higher visual 
workloads. Estimating the visual workload differ- 
ences between conditions requires the use of a side 
task which can be set to various levels of difficulty. 

Pilot Strategy and Role 

Many times experiments are performed to quan- 
tify how pilots normally do some type of activ- 
ity, for example, where pilots look while landing or 
where they look on takeoff. The following scanning- 
behavior analytical techniques have been successfully 
used to answer these types of questions: time histo- 
ries, time-locked time histories, and transition matrix 
measures. 

Time histories. If it is known ahead of time that 
a specific strategy is to be quantified (for instance, 
how many times a pilot transitions from the window 
to the instruments on a landing approach), then the 
researcher may skip the process of actually looking 
at the time histories and obtain the analyzed data 
directly from a computer program. However, it 
is good practice to look at the time histories to 
determine if some unexpected event may have taken 
place which relates to the pilot’s strategy and would 
otherwise be left unevaluated. 

The time history technique was useful in one in- 
vestigative study by the authors to identify which 
instruments pilots use during takeoffs. Particular 
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emphasis was placed on scan patterns of engine in- 
struments during takeoff. For the purposes of these 
tests, the lookpoint data (instrument being moni- 
tored) were plotted as time histories, and the pat- 
tern of engine instrument usage was evaluated by vi- 
sual inspection. Figure 12 shows such a time history. 
The 10 instruments at the top of the ordinate are 
the engine instruments in question. In the general 
pattern, the pilot looked at the engine instrument 
cluster three to four times during a takeoff. During 
the first look, he verified that the instruments were 
responding smoothly and normally to a small throt- 
tle increase. The second look occurred when takeoff 
thrust was applied. This second look sometimes in- 
cluded only the instrument monitoring engine pres- 
sure ratio (EPR) because the pilot was aiming for 
a specific EPR reading to take off. The third look 
was made just before the airplane reached 60 knots. 
With this look, the pilot verified that takeoff power 
had been set to the proper EPR value. There may 
or may not have been a look at the instrument mon- 
itoring engine speed. Finally, there may or may not 
have been one more look at the engine instruments 
before takeoff rotation. 

Time-locked time histories. A second analysis 
tool that has been useful is the time-locked time 
history. As seen in figure 4, there seems to be 
a definite strategy on what to look at after the 
secondary task is over: first a blink (short out-of- 
track), then a look at the attitude indicator, and 
then a look at the directional gyro. Pilot differences 
in scanning strategy could also be revealed with this 
analysis technique by comparing the pilots’ patterns 
of instrument usage. 

Transition matrix measures, A third analysis tool 
that has been useful is the transition matrix. Dwell 
percentages could show increased or decreased use 
of a particular instrument. Transition percentages 
could show changes or additions to normal or particu- 
lar patterns used to scan the instruments. Figure 13, 
taken from reference 19, shows the dwell times, dwell 
percentages, and transition percentages of two types 
of landing approaches flown with electronic attitude 
and horizontal situation indicators. One approach 
was a straight-in landing approach, and the other was 
a curved, descending approach. Not only are there 
significant differences in the transition percentages 
between instruments, but there is also a new tran- 
sition path with the curved approach between the 
electronic horizontal situation indicator and the air- 
speed indicator. This new transition path indicates 
that a new arrangement of the panel with a side- 
by-side arrangement of the electronic displays might 


not be a good design because of the increased phys- 
ical separation between the airspeed display and the 
electronic horizontal situation indicator. 

Pilot Training 

In some simulator studies, situations arose in 
which the experimenter felt the pilot was not using 
the instruments properly. When questioned about 
their use of the instruments and shown the video 
playback, the pilots usually agreed that an improve- 
ment , could be made by modifying their scanning 
behavior. In another study (ref. 17), both trainees 
and instructors were able to apply scanning-behavior 
data beneficially in a formal pilot training program. 

Video playback. A pilot training program was 
conducted for LaRC by Old Dominion University re- 
searchers at Piedmont Aviation. Piedmont instruc- 
tors and transition trainees were used in the pro- 
gram (refs. 20, 21, and 22). The results indicated 
that qualitative data obtained from a video tape 
telling pilots how they scan had some value and that 
scanning-behavior feedback from the instructors or 
from the trainees viewing their own scanning behav- 
ior can be a very useful tool for pilot training. The 
degree of usefulness of the scanning-behavior feed- 
back is a function of the trainee’s needs in establish- 
ing a good mental picture of the task to be accom- 
plished. The instructor pilots found the availability 
of real-time scanning data useful during the train- 
ing session in which flight technical errors were en- 
countered. The transition trainee pilots found that 
studying the video playback was helpful in correcting 
errors and in developing general piloting skills. 

Dwell histogram. One way a pilot learns is by 
watching the instruments to see their responses to 
his inputs. As he learns what to expect from the 
instruments, he no longer has to look at them as 
long. This learning process should show up in the 
average dwell time and in either the dwell histogram 
or the fixation histogram as shorter values for the 
peaks and fewer long dwells. 

Entropy rate. Preliminary analysis indicates that 
the entropy rates of trainees start very low (quite 
orderly) but approach the level of instructor pilots 
at the end of the training session. In other words, 
the trainees’ scan patterns are very structured at 
first because of unfamiliarity with the task, but as 
they learn the task, they rely more on reflexes or a 
“mental picture” of the way the aircraft will function. 
Consequently, they do not have to deliberately think 
about each item or step. Their scan patterns begin 
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(b) Curved approach. 

Figure 13. Dwell times, dwell percentages, and transition percentages for curved and straight-in approaches. 
Absence of data indicates absence of pilot scan. 
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to “loosen up”, that is, they are able to observe 
secondary information (more random scanning) and 
still fly the airplane satisfactorily. This analysis 
demonstrates that there is a significant amount of 
visual workload involved in the learning process and 
that the subjective impression of a higher workload 
for a novice is based upon the fact that the learning 
process affects the workload. 

Concluding Remarks 

This report documents the state of the art of ocu- 
lometric data analysis techniques and their applica- 
tions in certain research areas such as pilot workload, 
transfer of information from displays to pilots, strat- 
egy and role of pilots, and pilot training. These tech- 
niques include real-time viewing of the pilot’s scan- 
ning behavior, average dwell times, dwell percent- 
ages, instrument transition paths, dwell histograms, 
and entropy rate measures. Workload estimates are 
obtained from dwell percentages and entropy rate 
measures. Information transfer evaluations are per- 
formed primarily with dwell times, dwell histograms, 
and entropy rates. Pilot strategy and role are deter- 
mined by using scan time histories, time-locked time 
histories, and transition matrix data. Pilot training 
evaluation uses real-time scanning data and entropy 
rate measures. Overviews of the experimental setup, 
data analysis techniques, and software are presented. 
A glossary of terms frequently used in pilot scanning 
behavior and a bibliography of reports on related re- 
search sponsored by NASA Langley Research Center 
are also included. 

NASA Langley Research Center 
Hampton, VA 23665-5225 
March 28, 1986 
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Appendix A 

Overview of Oculometer System 

Daniel W. Burdette 
PRC Kentron, Inc. 

Hampton, Virginia 

Development of Oculometer 

The oculometer system used in simulation and laboratory studies at LaRC was a highly 
modified version of the Honeywell Mark 3A remote oculometer (refs. A1-A3), which allows 
head movements by a subject of up to 1 ft^. Basically, the system operates by projecting a 
beam of collimated infrared light at one of the subject’s eyes (fig. Al). Two reflections from 
the eye are returned to a video camera. The first is a broad (4- to 8-mm) reflection of the 
retina, bounded by the pupil, like a cat’s eye reflecting from the headlight of a car; the second 
is an intense pinpoint reflection from the surface of the cornea. From the video signal of 
the eye’s reflections, the computer determines the center of each reflection. Based upon the 
relative positions of the reflections, the computer, calculates the pilot’s foveal lookpoint on the 
instrument panel. A video tape of the instrument panel and the pilot’s superimposed lookpoint 
is saved as a permanent record of the test. The lookpoint coordinates and pupil diameter are 
recorded for subsequent computer analysis. Recordings of the aircraft state variables, pilot 
control inputs, and other measures of interest can be recorded at the same intervals for use in 
correlating pilot lookpoints with flight conditions and other factors. 

Hardware modifications. Several hardware modifications were made to the original 
Honeywell system to increase the utility of the oculometer system. The primary modification 
was made to the electro-optic (EO) head. Figure A2 shows the EO head as received from 
Honeywell. The TV camera inside the EO head was replaced by a smaller one, and internal 
wasted space was eliminated. The resulting EO head (fig. A3) was about one-third the size 
of the original head. The next change was the addition of analog output channels so that the 
oculometer could output not only the lookpoint coordinates but also a voltage proportional 
to the instrument being observed at by the subject. This feature allowed easier recording of 
data by external microcomputers, such as the one in figure A4. The infrared light source 
and collimation scheme were modified next so that a 15-W bulb could replace the 150-W 
projection bulb. This modification eliminated the need for a large heat sink and cooling fan and 
further reduced the size of the EO head to its current size of about one-fourth the original size 
(fig. A5). This size reduction was enough to allow the EO head to be placed behind many 
simulator instrument panels (fig. A6). The last hardware modification was the addition of a 
single-chip microprocessor to process eye data in parallel with the oculometer computer. Its 
only function was to position the two tracking mirrors to keep the eye’s reflections in the center 
of the TV image. The effective tracking speed of the mirrors increased by over a factor of two, 
and the tracking time of the oculometer on a typical run improved from about 75 to 95 percent. 
The remaining out-of-track time is caused by subject eye blinks and a very rare loss of track 
due to excessive head movement. 

Software. A number of oculometer software modifications were made to the original 
Honeywell oculometer software to make it more user friendly and versatile. Software was added 
that would automatically adjust calibration coefficients based upon lookpoints collected while 
the object looked at several (up to 28) calibration points. Improved eye center determination 
routines were developed that would minimize the effects of upper and lower eyelid distortions 
of the returned pupil reflection. Provisions were also made for instrument panels of complex 
geometries, such as those with flat panels at an angle to the main panel or panels located 



in front of or behind the main panel. Finally, better terminal communication routines were 
written to allow the operator to interact in real time with the oculometer computer instead of 
having to stop the computer from processing eye data and go into the communication mode 
exclusively. 

The volume of data generated during a test makes computer processing a necessity. 
Initially, no software for data recording and reduction was available for scanning behavior. 
Software programs have been developed that will process the data into standardized formats. 
(Appendix B describes three of these programs in more detail.) Inputs to the program 
are instrument boundaries and conditional flags to direct the type of processing and output 
generation. The output data from these programs can later be summarized and statistically 
tested. Every attempt has been made to standardize the data recording procedure to minimize 
the software changes required from test to test. However, every study has required software 
changes because of unique test goals, designs, simulator/aircraft configurations, and other 
constraints. 


Outline of Hardware and Procedures for Current Oculometer System 

I. Functional description of hardware subsystems and components 

A. EO head 

1. A beam of parallel light rays (wavelength of 800 to 900 nm) is produced by several 

internal components: a tungsten filament source (6 V/15 W), an IR filter to 

remove most visible light, a heat filter to prevent unwanted far-infrared energy, two 
collimating lenses, and a beam splitter. 

2. The moving mirror assembly contains separate azimuth and elevation mirrors to direct 
the IR beam toward the eye of a test subject. The servo system can be manually 
controlled, operator assisted through the use of a computer-interfaced joystick, or 
driven solely by either the Nova computer or a separate and dedicated microprocessor 
circuit. 

3. Image collection optics, including the positive lens, the servo-positioned negative lens, 
and folding mirrors combine to focus the eye image onto the surface of a silicon 
matrix vidicon tube. The motorized positioning system for the negative lens is 
mechanically coupled to a multiturn potentiometer which provides focus distance 
information necessary for lookpoint calculations. 

4. A Dage-MTI television camera (black-and-white) model SC-21 with a horizontal 
resolution of 450 TV lines and a vertical resolution of 375 lines is used. The 
camera contains a General Electric type Z7996A silicon matrix vidicon tube (1 in.), 
which efficiently responds to the near-infrared wavelengths present in the focused eye 
image. The television picture is further enhanced through the use of nonstandard 
adjustments to the camera control unit, which produce maximum contrast between 
the pupil and cornea reflections while reducing nonessential details of the eye and 
skin to the video black reference level. The resulting eye image forms the basis for 
the entire oculometer system, and it is from this one analog signal that all subsequent 
data are derived. 

B. Oculometer control unit 

The oculometer control unit (LaRC design) provides manually variable alternating- 
current (AC) voltage to control the intensity of the tungsten filament light source, control 
voltages for azimuth and elevation mirrors used to follow the eye, and adjustment voltages 
to the scan converter for matching the scene camera video to the ideal point and lookpoint 
output. (See section III. A. 5.) 



C. Closed-circuit television 

1. Video cameras and monitors 

a. The EO head camera operates in conjunction with a rack-mounted control unit. 

b. The cockpit scene camera (in sync with the EO camera) is used to provide a 
picture of the cockpit instrument panel upon which lookpoints and ideal points can 
be superimposed. This video presentation transforms the rather abstract analog 
voltage outputs from the oculometer into a real-world and real-time picture of eye 
scanning behavior. 

c. The head tracker camera gives the operator a view of the subject’s face for coarse 

positioning of the IR beam when using either the manual control knobs or the 
joystick. 

d. Triple 5-in. black-and-white monitors are used for simultaneous viewing of all 
three video signals. 

2. Video signal processing devices 

a. A combination video data insertion generator/sync stripper provides horizontal 
and vertical drive signals to the interrupt-driven oculometer program, as well as 
to the scene camera. In addition, a small insert of the eyeball portion of the EO 
video is placed into the scene video for tape recording and subsequent diagnostic 
review, if necessary. 

b. The sync delay unit works in conjunction with the above device to position the 
eye image insert. 

c. The video tape recorder is a 1/2 in. reel-to-reel model. 

d. A video timer superimposes the date and time on the combined video signals for 
later reference. 

e. The scan converter is a type of storage oscilloscope which displays the analog 
outputs from the Nova computer in graphic form and superimposes the lookpoint 
dot or ideal point dots onto the combined scene television picture. 

f. The television waveform monitor is an oscilloscope specifically designed for viewing 
composite television signals. 

D. Computer system 

1. The Nova 800 computer system accepts video data in analog form, digitally calculates 
a lookpoint, and provides analog voltages representing x and y coordinates of the 
lookpoint, as well as pupil diameter data, azimuth and elevation mirror commands, 
and error codes to help diagnose out-of-track conditions. 

2. The Intel 8751 signal-chip microcontroller, working in conjunction with the Nova 
computer and assembled as a plug-in board for the Nova card cage, provides improved 
real-time tracking of a moving eyeball and relieves the main computer of a time- 
consuming task. 

3. The computer terminal allows operator interaction with the program and provides 
hard copy through its thermal printer. 

II. Setup procedures 

A. Preliminary site survey 

1. Outside cockpit 

Choose a location for the system rack, taking into account AC power requirements 
(110 V/15 A) and cable routing between the rack and the cockpit. Dedicated cables 
exist at LaRC between some flight simulators and a central oculometer laboratory. A 
single AC power line should be used for all oculometer equipment to avoid electrical 
noise. In some simulators it has been necessary to magnetically shield and electrically 
insulate components from interference within the cockpit. 



2. Inside cockpit 

a. Decide on the test subject’s visual area of interest within the total instrument 
panel (typically limited to approximately 40 in, horizontally by 20 in. vertically). 

b. Choose an EO head location from several possibilities. The following factors 
should be considered: 

(1) The mirror box assembly must fit into the simulator with minimum disruption 
of the normal instrument panel configuration. 

(2) Optical paths should be as short as possible and should avoid the chance of IR 
beam obstruction during any phase of the flight simulation. 

(3) The mirror box must be near the bottom of the area of interest to avoid eyelid 
obstruction of the pupil. Any glances below the level of the mirror box will 
inevitably cause the subject’s upper lid to droop. 

(4) The blind spot of the eye should be avoided, since the pupil return is greatly 
diminished. For example, if the left eye must be avoided in a particular cockpit 
because of control yoke blockage of the light beam, avoid placing the mirror 
box such that the IR beam will strike the blind spot in the subject’s right eye 
each time an important instrument is viewed. 

(5) The face of the mirror box should be coincident with the fixation plane to 
minimize large geometrical compensations in the program. 

c. Choose a location for the scene camera to provide an undistorted and unobstructed 
view of the area of interest on the instrument panel. 

d. Choose a location for the head tracker camera that gives a clear view of the 
subject’s face without causing a great deal of visual distraction. 

e. Plan to locate the mirror amplifier within 10 ft of the EO head to minimize cable 
losses. In motion-based simulators all hardware must be mechanically secure. 

B. EO head configuration and calibration 

1. Orient the vidicon mechanically and electrically such that with the camera pedestal 
control turned up, a focused image (such as that of a ruler) appears on the monitor as 
correct (i.e., right side up and nonreversed horizontally). Minimize optical reflections 
within the EO head by coating bright surfaces with optically flat black paint. 

2. While observing the composite video output on the TV waveform monitor, decrease 
the camera pedestal control until a clear image of an artificial eye is produced with 
the pupil reflection at approximately 0.5 V and the cornea reflection at 1.0 V. Also 
insure that the image is free of extraneous video information due to optical reflections 
or electrical noise, since such artifacts can cause the moving-mirror servo system to 
track false targets. Other potentially damaging effects include erroneous data output 
and complete loss of track during system operation. 

3. Measure the image collection optics and enter scaled machine values of the distances 
and focal lengths into the program by using the multiplication factors and computer 
symbols listed in table AI. 

4. Center the light source for maximum pupil return of an artificial eye as the image is 
moved across the center of the vidicon by manually controlling the mirrors. The beam 
splitter should be positioned such that the collimated light beam and the television 
camera share a common optical axis. 

5. Measure the angles at the zero position of the azimuth and elevation mirrors relative 
to the mirror box. Any variation from the nominal 45® for each must be entered into 
the program in the form of direction cosines or mirror box pitch-up. Notice that a 
mirror position offset of 1® results in a deflection of 2® of the IR beam. Of course, 
the mirrors must be able to rotate sufficiently to track an eye in any location within 
the cubic foot of space. Section II.C contains more information on the oculometer 
program geometry. 



6. Measure the IR irradiance of the eyespace (30 in. from the face of the mirror box). 
Vary the intensity, and note the voltage when irradiance reaches 1.67 mW/cm^ (10.8 
mW/in2), the upper limit allowable for use at LaRC. A more detailed discussion of 
safety procedures is contained in section III.D of this outline. 

C. Geometry 

1. With the EO head installed in the cockpit, establish a reference point on the fixation 
plane. Express any offsets between this point and the origin of the oculometer axis 
system as XO, YO, and ZO in the program. See table All(a) for the equations used to 
calculate the offsets. The origin of the oculometer axis system is the point at which 
the center of the IR beam intersects the face of the mirror box when the mirrors 
are at their null position (0,0 volts). A convenient way to choose the fixation plane 
reference point is to look directly into the IR beam with the mirrors at their null 
position. Then by extending this line of sight through the mirror box, the reference 
point can be marked on the instrument panel for use in later measurements. Since 
this point lies within the oculometer Z-axis, the horizontal and vertical offsets are 
equal to zero (XO = 0, YO = 0). The symbol ZO equals the distance in inches between 
the mirror box face and the fixation plane reference point multiplied by 256. When 
the fixation plane lies behind the mirror box, ZO is negative. 

2. Calculate and enter the constants representing mirror box pitch-up and fixation plane 
direction cosines. (See tables All(b) and AII(c).) First, measure the angle between 
the oculometer Z-axis (a line directly along the IR beam with the mirrors at their 
null position) and the fixation plane Z-axis (a line perpendicular to the instrument 
panel originating at the reference point). Then use this angle to determine which of 
the following three calculation methods is appropriate: 

a. The angle between the two axis systems is zero. In this ideal installation, the 
mirrors are mounted in the mirror box at their nominal position of 45°, and the 
IR beam that emerges is perpendicular to both the mirror box and the instrument 
panel. There is no mirror box pitch-up (D1 = 0, D2 = 16384), and all direction 
cosine constants have the following nominal values: 

Q1 = 1024 Q2 = 0 L3 = 0 

Ml == 0 M2 = 1024 M3 = 0 

N1 = 0 N2 = 0 N3 = 1024 

b. The angle has a vertical offset resulting from a pitch-up of the mirror box relative 
to the instrument panel. All direction cosine constants remain nominal as in the 
previous example. 

c. The angle has both vertical and horizontal offsets. Once the pitch and yaw 
components of this angle have been determined, the values for the direction cosine 
constants can be calculated by using the equations given in table AII(c). For 
simplicity, it is best to assume in this case that the pitch-up angle of the mirror 
box is equal to zero (Dl = 0, D2 = 16348). In all direction cosine calculations, 
the roll angle can be reduced to zero by properly rotating the EO camera vidicon 
as described in Section II.B.l. 

3. Choose a digital-to-analog conversion scale faetor such that an output of ±5 V from 
each lookpoint channel is sufficient to cover the visual area of interest (table All(d)). 
For example, a scale factor of 0.25 V/in. allows coverage of an instrument panel 
approximately 40 in. by 40 in. (SCFX2 = 1638, SCFY2 = 1638.) 

4. Specify the desired output voltages when the subject looks at the infrared light source 
by calculating and entering machine values for DDX and DDY according to the 
equations shown in table All(e). The purpose of DDX and DDY is to allow the 
operator to set the voltage output when the subject looks at the EO port. Since the 



EO port is not usually located at the center of the instrument panel, these values 
also define the location of the point on the instrument panel which corresponds to a 
system output of zero volts in both the x and y channels. Careful choice of DDX and 
DDY can also help to avoid exceeding the analog output limitations of ±5 V when 
the subject looks at instruments located at relatively large distances from the center. 

5. Specify CALX and CALY coordinates such that approximately three fourths of the 
total visual angle is covered in each direction from the instrument panel reference 
point (table All(f)). 

6. Establish the fixation plane ideal points to be used in both the start-up and eye 
calibration phases of system operation. In a typical flight simulator, it will be 
necessary to measure and note the horizontal and vertical distances between the 
center of each instrument and the fixation plane reference point. Enter scaled values 
of these measurements as appropriate (TX)n and (TY)n constants for the various 
instruments. (See Table All(g).) 

7. Choose a home base point, such as the center of the attitude indicator (flight 
director), and set XCNTR and YCNTR for quick adjustment of the lookpoint during 
experimental sessions. (See table All(h).) Switch no. 13 on the front panel of the 
Nova computer can force the lookpoint to this position at any time. When used 
judiciously by a skilled oculometer operator, this feature can greatly improve the 
accuracy of the data output. 

8. Fine tune the description of EO head optics by adjusting computer constants FAO and 
RSO according to the successive approximation formula provided in volume 1 of the 
operating manual for the oculometer (ref. A3). Adjust constant L9 so that a constant 
pupil diameter output is obtained when an artificial eye is focused at various points 
within the cubic foot of eye space. After L9 has been adjusted, print out the values 
MAG and CMAG. Change CMAG according to the equations shown in table AI. Use 
the printed value of CMAG as CMAGqld- 

9. Verify the accuracy of these steps by checking E3 for several eye focus distances. The 
symbol E3 represents the distance from the fixation plane to the subject’s eye. Also 
check for proper lookpoint output for various horizontal and vertical rotations and 
translations of both an artificial and a human eye. 

III. Operating procedures 
A. Start-up 

1. Turn on the power to the main equipment rack, the mirror amplifier, and the scene- 
and head-tracking cameras. 

2. Set switch no. 10 on the front panel of the Nova computer up (all others down) and 
start the oculometer program by activating the “stop,” “reset,” and “start” control 
switches. Three calibration points should be visible on the scan converter. (Increase 
the intensity if necessary.) 

3. Set ZOFFX and ZOFFY to 0 by typing the name of each constant followed by a slash 
(/) on the terminal keyboard. After each stored value is printed by the program, 
simply type in the new value. Numerical values are always considered to be octal 
unless followed by a decimal point. 

4. Set switches no. 6 and 7 up to generate ideal points on the scan converter and the 
scene monitor. Use the storage mode of the scan converter to freeze these dots on 
the scene monitor. Whenever adjustments are made, the previous positions should 
be erased. 

5. Align the ideal points with their corresponding locations on the instrument panel 
as represented on the scene monitor. Distortions can be reduced or eliminated by 
adjusting the appropriate potentiometers for vertical or horizontal gain, bias, and 
cross talk. The video systems and oculometer program are now internally calibrated 



such that a given lookpoint output from the computer will appear at the proper 
location on the video image from the cockpit scene camera. It should be noted that 
these corrections do not alter the output of the oculometer computer, but they simply 

aid m presenting the data in a form which improves system operation, calibration, 
and interpretation. ’ 

6. Verify all data links between the oculometer system and the data collection comput- 
ers. System output data can be artificially manipulated by a combination of software 
and hardware adjustments. 

7 . Set the date and time on the video timer. 

B. Eye calibration 

1. Enable the servo-driven mirror system and set switch no. 6 up on the Nova front 
panel to begin tracking the chosen eye of the test subject. It will be necessary to 
adjust the IR beam intensity for a good video signal. Verbally direct the subject’s 
vmw to the instrument panel home base, usually located at the center of the flight 
director. Force the lookpoint output to this position by flipping switch no. 13 up 
and down. Voltage offsets ZOFFX and ZOFFY are thus added to the oculometer 
digital-to-analog outputs. The scene monitor should now display a small dot in the 
center of the flight director. Decrease the intensity of the scan converter until the dot 
disappears when the eye is out of track (e.g., during each eye blink). 

2. As the operator verbally directs the subject’s gaze to various instruments on the 
panel, the linearization coefficients (see table AIII) can be adjusted to correct for any 
distortions in the lookpoint output due to the eye itself. Unpredictable results can 
occur if this procedure is attempted before the internal calibrations are completed. 
(See section III.A.5 of this outline.) 

3. Observe the electrical representation of the video signals from the pupil and cornea 
as seen on the waveform monitor while the subject scans the instrument panel. Large 
changes in amplitude for various look angles indicate that the reflectivity of the retina 
is inconsistent and suggest that an automatic intensity controller be used for a more 
steady IR return. 

4. Adjust the servo-driven mirror control system for push-button return to a nominal 
eye position by reading values of MCX and MCY at an appropriate moment and 
then inserting these values into computer memory locations MCXHL and MCYHL, 
respectively. The constants MCX and MCY represent actual mirror command 
voltages sent to the yaw and pitch mirrors, respectively. 

5. Print the values of all linearization constants as well as both ZOFFX and ZOFFY 
for later reference. Saving a hard copy of this information for each test subject 
simplifies future calibrations and improves the efficiency of system operation during 
test sessions. 

C. Data collection 


1 . 

2 . 

3. 

4. 

5. 


Prior to the start of the simulation scenario, set switch no. 6 up, enable the mirror 
servos, and adjust the IR level to begin tracking the test subject’s eye, which must 
be the same eye as that used for calibration. 

Recheck the offsets by requesting a brief, but steady, glance at the home base of the 
instrument panel. Corrections may be made either by typing the previously listed 
values of ZOFFX and ZOFFY or simply by using switch no. 13 at the proper time. 
Start the video tape recorder at the beginning of each period of data collection. 

At the conclusion of each run, stop the video recorder, decrease the IR intensity, and 
set switch no. 10 up to place the computer in a pause mode. 

At the end of the daily session, set switch no. 10 up and stop program execution with 
the “stop” switch. Turn off the main rack circuit breaker and all other equipment. 
Record essential details of the session in the oculometer log. 



D. Safety 

Although the IR level required for good oculometer operation is quite low, LaRC 
maintains certain standards for eye safety. The maximum irradiance measured at 
an eyespace 30 in. from the face of the mirror box should not exceed 1.67 mW/cm^ 
(10.8 mW/in^). This measurement must be performed at least once for each 25 hours 
of oculometer operation and entered into a permanent log. The log entries, which are 
initialed by the system operator, include such relevant information as the location of the 
installation, the name of the test subject, and the total daily time in use. As a further 
precaution, baseline and semiannual eye examinations are required for each test subject. 
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Symbols and Abbreviations 


ASCII 

American Standard Code for Information Interchange 

CALX, CALY 

distances from reference point to arbitrary horizontal and vertical 
calibration points on the fixation plane 

CMAG 

constant used to adjust the value of MAG 

DDX, DDY 

offsets used in specifying a zero-volt output from both x and y data 
channels 

EO 

electro-optic 

IR 

infrared 

MAG 

magnification of oculometer image collection optics 

ZOFFX, ZOFFY 

computer constants representing voltage offsets used to force the ocu- 
lometer system output to the chosen home base point on the instrument 
panel 



TABLE AI. CONVERSION OF EO HEAD DIMENSIONS TO MACHINE VALUES 


(a) Definition of parameters 


Parameter" 

Multiplication 

factor 

Program 

symbol 

Focal length of positive lens 

1024 

FI 

Focal length of negative lens 

1024 

F2 

Distance between camera face and positive lens 

1024 

LI 

Distance between negative lens and camera face 

1024 

FAO 

Distance between yaw mirror and positive lens 

256 

RSI 

Distance between yaw mirror and pitch mirror 

256 

RS2 

Distance between pitch mirror and electro-optic port 

256 

RS3 


®A11 measured in inches. 


(b) Magnification equations 


pA/r A n _ _ Calculated magnification 

NEW mag 


X CMAGold 


where 


Calculated magnification = 256 


-(EA0)2 + FAO(Ll-Fl) + F2{L\ - FI) 




TABLE AIL FIXATION PLANE CONSTANTS 
[All distances measured in inches] 

(a) Fixation plane origin offsets with respect to oculometer axis system 

XO = 256 X (X-axis offset) 

YO = 256 X (y -axis offset) 

ZO = 256 X (Z-axis offset) 


(b) Mirror box pitch-up 

D1 = 16384 X sin(Pitch-up angle) 
D2 = 16384 X cos(Pitch-up angle) 


(c) Direction cosines of fixation plane 

X-axis: Q1 = 1024 cos iJj cos 6 

Ml = 1024 (cos 'tp sin 6 sin 4> — sin V’ cos (f>) 

N 1 = 1024 (cos tp sin 6 sin (p + sin ip cos <p) 

Y-axis: Q2 = 1024 sin ip cos 6 

M2 = 1024 (sin ip sin Osin^ + cos ip cos 4>) 

N2 = 1024 (sin ip sin 6 cos (p — cos ip sin 0) 

Normal: L3 = -1024 sin 61 

M3 = 1024 cos 0 sin (p 
N3 = 1024 cos 0 cos 0 

where 

Ip = Roll angle 
9 - Yaw angle 
(p = Pitch angle 

(d) Scale factors 

SCFX2 = 6554 x (Volts per horizontal inch on instrument panel) 

SCFY2 = 6554 X (Volts per vertical inch on instrument panel) 

(e) Location of zero- volt output from x and y data channels 
SCFX2 

DDX — X (Horizontal distance from reference point to zero-volt 2 i-channel output) 

64 

SCFY2 

DDY = X (Vertical distance from reference point to zero-volt y-channel output) 

64 
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TABLE AIL Concluded 

(f) Calibration points 

CALX = 16 X (Distance from reference point to horizontal calibration point) 
CALY = 16 X (Distance from reference point to vertical calibration point) 

(g) Ideal points 

(TX)„ = 8192 X distance from reference point to center of instrument n) 

(Distance from horizontal reference point to calibration point) 

^'j'Y) = 8192 X distance from reference point to center of instrument n) 

(Distance from vertical reference point to calibration point) 

(h) Home base point 


XCNTR = 
YCNTR = 


'SCFX2 ^ „ . 1 

04 ^ (Horizontal distance from reference point to home base point) + DDX 

'SCFY2 ^ . 1 

04 ^ (Vertical distance from reference point to home base point) + DDY 
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Figure A2. Honeywell electro-optic head. 
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Figure A6. Current electro-optic heads installed in cockpit. 
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Appendix B 

Overview of Data Reduction 

The primary data reduction effort for oculometric 
studies involves the use of three programs: SCAN, 
SUMMARY, and HISTO. These programs were 
developed specifically to enhance oculometer data 
reduction. 

Functional Description of Program SCAN 

Program SCAN calculates raw scanning-behavior 
parameters. It is by far the most important of the 
three primary oculometric programs, since it creates 
files to be used by the other programs and provides 
an initial detailed analysis of oculometric data. Fig- 
ure B1 presents a very simplified flowchart of pro- 
gram SCAN which can be referred to in the follow- 
ing discussion. SCAN’s input is simple and consists 
of flag settings to control program options, instru- 
ment boundary coordinates, and binary-coded data 
file(s) containing time histories of oculometric and 
other data (generally aircraft state and performance 
data). The output is a comprehensive first look at 
scanning-behavior data for each individual run and 
includes lookpoint (Markov) transition matrices, his- 
tograms of dwell time occurrences versus length of 
occurrence for each instrument, counts of the num- 
ber of control inputs made by the pilot, means and 
standard deviations of instrument dwell times, state 
and performance variables, and other single-variable- 
statistics. SCAN also outputs two ASCII coded files, 
one used by program SUMMARY and the other by 
program HISTO. The file for SUMMARY is a com- 
pilation of the single-run output statistics of SCAN. 
The file for HISTO contains a sequential listing of 
the instrument number, the length of the fixation, 
and the number and timing of control events (if any) 
which occurred during that fixation. Special purpose 
modifications to SCAN have been made in the past 
to recreate graphic displays for determining at which 
individual element of the graphic display information 
the pilot was looking. 

Functional Description of Program Summary 

Program SUMMARY, as the name implies (see 
fig. B2), combines and summarizes most of the single- 
run statistics that are output by SCAN. That is, after 


a complete data set is finished, SUMMARY can be 
executed with the multiple-run files from SCAN to 
provide a data set summation. In addition to the 
summation of SCAN information, SUMMARY also 
performs the necessary computations required for t 
tests and F ratios on instrument scanning-behavior 
parameters and aircraft state and performance vari- 
ables. The statistical comparisons are controlled by 
user input flags and include pilot versus pilot, condi- 
tion versus condition, and pilot versus condition. A 
maximum of six comparisons can be made in a sin- 
gle execution. The output of SUMMARY consists of 
four tables. Each table is a listing of the calculated 
statistics for each of the up-to-six comparisons for 
all the scanning-behavior data and the aircraft state 
and performance variables. The first table contains 
the F ratios, the second table contains the variances, 
the third table contains the t statistics and degrees 
of freedom, and the fourth table contains the means. 

Functional Description of Program HISTO 

Program HISTO calculates dwell time distribu- 
tions. It is executed with the file(s) of fixation times 
and control input event times from program SCAN. 
Figure B3 illustrates the basic operations performed 
to obtain the dwell histograms. HISTO reduces these 
data into dwell time histograms. Six tables are pro- 
duced with the data. The first four are dwell time 
histograms, and the last two are fixations/dwell his- 
tograms. The first histogram represents the number 
of times that a subject’s dwell length during con- 
trol inputs lasted a designated time for a given in- 
strument. The second converts the counts of the 
first histogram to a percentage with respect to to- 
tal counts for that particular instrument. The third 
and fourth histograms differ from the first two only in 
that no control input was made during these dwells. 
The fifth and sixth histograms depict the number 
of fixations per dwell for a given instrument with 
and without control inputs, respectively. HISTO can 
be modified to output fixation histograms instead of 
dwell histograms, since the data file contains fixa- 
tion counts. Special purpose modifications might 
include histograms of the time from the beginning 
of a dwell/fixation to a control input or histograms 
of the time from a control input to the end of a 
dwell/fixation. 
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Figure B3. Flowchart of HISTO computer program. 
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Appendix C 
Glossary 

Average dwell time The total time spent looking at an instrument divided by the total 
number of individual dwells on that instrument. 

Dwell percentage Dwell time on a particular instrument as a percent of total scanning 
time. 

Dwell time The time spent looking within the boundary of an instrument. 

Fixation A series of continuous lookpoints which stay within a radius of 1 visual degree. 

Fixations per dwell The number of individual fixations during an instrument dwell. 

Lookpoint The current coordinates of where the pilot is looking during any one thirtieth 
of a second. 

Oculometer A device which measures the lookpoint of a test subject. 

One-way transition The sum of all transitions from one instrument to another (one 
direction only) in a specified instrument pair. 

Out of track A state in which the oculometer cannot determine where the pilot is looking, 
such as during a blink or when the subject’s head movement has exceeded the tracking 
capabilities of the oculometer. 

Saccade The spatial change in fixations. 

Scan Eye movement technique used to accomplish a given task. Measures used to quantify 
a scan include (but are not limited to) transitions, dwell percentages, and average dwell times. 

Transition The change of a dwell from one instrument to another. 

Transition Rate The number of transitions per second. 

Two-way transition The sum of all transitions between an instrument pair, regardless of 
direction of the transition. 
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